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and less often solved correctly for all groups. of subjects (except 
high algorithm/high conceptual groups) than the corresponding 
algorithmic-mode problem although the algorithmic-mode problem took 
longer and required more transitions between episodes. Correct 
conceptual understanding, as reported by the problem solvers on 
transcribed think-aloud interviews, rested upon a known definition 
and an increased understanding of the language of chemistry. 
Suggestions are given to improve students' conceptual base in 
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Assessing Student Problem-Solving Success on Selected Topics in 

Introductory Chemistry 



Abstract 



Incident identification graphs can be used to diagnose areas of difficulty in a subject's 
problem-solving schema at the episodic level. In this study 22 subjects (two experts and twenty 
novices categorized into five problem-solving groups (expert, HA/HC, LA/HC, HA/LC, and LA/LC) 
were interviewed whiie solving problems on four topics generally found in introductory-level 
college chemistry (i.e., density, stoichiometry, bonding, and gas laws). Of these topics the one 
which presented all categories of problem-solvers the most difficulty was the paired algorithmic- 
conceptual question on stoichiometry. Consequently, this paired problem was investigated in 
greater depth. Further analysis yielded that the conceptual-mode problem of the paired question 
was far more difficult and less often solved correctly for all groups of subjects (except HA/HC) than 
the corresponding algorithmic-mode problem although the algorithmic-mode problem took longer 
and required more transitions between episodes. Correct conceptual understanding, as reported 
by the problem solvers on transcribed think-aloud interviews, rested upon a known definition and 
an increased understanding of the language of chemistry. At the episodic level of the problem- 
solving schema, the time needed to read a problem for the LC groups was almost twice the time 
needed for the expert and HA/HC groups. Suggestions are given to improve students 1 
conceptual base in introductory chemistry. 



Purpose 

Obvious differences were seen in the problem-solving schema representative of different 
categories of problem solvers when algorithmic and conceptual problems typically found in 
introductory-level chemistry courses were solved (Mason & Crawley, 1994). This investigation 
sought to establish a comparison between a subject's problem-solving ability and the similarities 
and differences of their approach to solving stoichiometric problems common to many 
introductory-level chemistry courses. The central question is: what are the similarities and 
differences in the problem-solving schema used by different groups of problem solvers as they 
solve paired algorithmic and conceptual problems regarding stoichiometric relationships in 
chemical reactions? 
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Theoretical Base 

The main focus of this study was to examine the similarities and differences in the ways 
different categories of problem solvers found solutions to problems from different topic areas (i.e., 
density, stoichiometry, bonding, and gas laws). In this study comparisons were made between 
different categories of problem solvers at the episodic level of their respective problem-solving 
schema as they solved paired problems. Novices were classified into groups reflective of their 
problem-solving ability (Mason & Crawley, 1994; Nakhleh, 1993); namely, high algorithmic/high 
conceptual (HA/HC), high algorithmic/low conceptual (HA/LC), low algorithmic/high conceptual 
(LA/HC), and low algorithmic/low conceptual (LA/LC). The episodes chosen to represent the 
problem-solving schema were that of read, define, set up, solve, and check. The selection of 
these specific episodes was based on previously published heuristics (Bodner & McMillen, 1986; 
Bunce, Gabel, & Samuel, 1991; P6lya, 1945; Powers, 1984; Schoenfeld, 1979, 1980; Tingle & 
Good, 1990; Woods, 1987, 1992). 

The four topics investigated (i.e., density, stoichiometry, bonding, and gas laws) also 
were chosen because of their frequency of occurrence in research literature and in introductory 
chemistry courses. Modifications of the selected paired questions on density, stoichiometry, and 
gas laws used in this study were previously investigated by Nakhleh and Mitchell (1993) and 
Mason and Crawley (1994). Only the topic of stoichiometry is considered in this research paper 
because the data collected from all paired questions support that this topic is far more difficult for 
both experts and novices than any of the other three topics (i.e., density, bonding, and gas laws), 
and therefore warrants a more detailed investigation. 

Novice problem solvers usually have difficulty in solving problems due to a lack of prior 
knowledge in a specific content area, not because they simply lack the ability to solve problems 
(Shuell, 1990a). Shuell (1990b, p. 532) reported that "learning is an active, constructive, 
cumulative, and goal-oriented process that involves problem solving.- Neural connections made 
during a learning situation either strengthen or weaken previously established connections. 



ERLC 



4 



Therefore, it is imperative that new inputs appropriately target prior knowledge by strengthening 
correct associations and by delating any misconceptions which may be held by a student. 

Novice problem solvers in chemistry usually have greater success with solving problems 
of a algorithmic-mode than problems having a more conceptual base (Bunce, 1993; Nakhleh, 
1993). Niaz and Robinson (1992) concluded that student training in computational problems did 
not guarantee successful understanding of conceptual problems. According to the study, Niaz 
and Robinson (1992) concluded that algorithmic and conceptual problems required different 
cognitive abilities. 

Design and Procedures 
Novices (n«20) were selected from a large-lecture university class (n*1S0) using stratified 
random sampling techniques (based on college enrollment) to solve paired algorithmic and 
conceptual problems by a think-aloud protocol along with two chemistry professors. Four 
different problem topics were investigated over the course of this one semester study. The 
topics selected included density, stoichiometry, bonding, and gas laws. Subjects were grouped 
into the various problem-solving categories based on the correctness of their responses to a 
selected problem (see Table 1). (A subject who succeeded at solving three or four problems 
correctly was classified as high algorithmic (HA) and high conceptual (HC) depending upon the 
. mode of the problem, and accordingly, a subject was placed into the low algorithmic (LA) and low 
conceptual (LC) categories if only one or two problems of a particular mode were solved correctly.) 

Table 1 

Distribution of novices within problem-solving categories for the experimental sample, n=20 





HA 


LA 


HC 


2 


0 


LC 


13 


5 



9 
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Next, comparisons were made from the interpretation of the incident identification graphs 
completed for each subject at the time of the think-aloud interviews on all selected topics. The 
interpretation of data from these graphs led to the selection of the paired stoichiometric problem 
for extended study. The incident identification graphs used in this study on stoichiometry were 
similar to those reported as examples reflective of the problem-solving schema of the typical 
member of each problem-solving category (Mason & Crawley, 1994). Comparisons made were 
based on the time required to complete the problems and the number of transitions needed to 
solve the problems. Selected variables were graphed on an x-y axis for further comparisons 
between the topics and for identification of specific problem areas in the problem-solving schema 
associated with stoichiometry. 

The paired stoichiometric problem used in this part of the investigation can be found 
below. This problem is a modification of a paired stoichiometric problem published by Nakhleh 
(1993). By dissecting the problem-solving schema of the 22 interviewed subjects, grouped 
according to their problem-solving category, the following research questions and hypotheses 
were tested. 

How do experts and high-ability algorithmic/high-ability conceptual (HA/HC), low-ability 
algorithmic/high-ability conceptual (LA/LC), high-ability algorithmic/low-ability conceptual 
(HA/LC), and low-ability algorithmic/low-ability conceptual (LA/LC) novices solve paired 

♦ 

algorithmic and conceptual problems on the selected topic of stoichiometry? (What are 
the similarities and differences between their approaches to solving paired algorithmic 
and conceptual problems?) 

Hi - There are differences among the problem-solving strategies of experts, HA/HC, 
LA/HC, HA/LC and LA/LC novices in the methods used to solve algorithmic- 
mode problems on the selected topic of stoichiometry in introductory chemistry . 

H2 - There are differences among the problem solving strategies of experts, HA/HC, 
LA/HC, HA/LC and LA/LC novices in the methods used to solve conceptual- 
mode problems on the selected topic of stoichiometry in introductory chemistry. 
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Example Stoichiometric Problem Selected for Interviewing 

A. Calculate the maximum weight of NH3 that could be produced from 1 .9 mol of hydrogen 
and excess nitrogen according to the following reaction: 

N2 + 3H2/E2NH3 

(A) 15 g 

(B) 28 g 

(C) 22 g 

(D) 30 g 

(E) 17 g 

C. Any quantity of Cu in excess of one mole will always react with two motes of AgN03 to 

produce one mole of Cu(N03)2 and two moles of Ag. Therefore we know that 1 .5 moles 
of Cu will react with two moles of AgNOa to produce 215.74 grams of Ag. Which of the 
following concepts is the only concept NOT associated with all balanced equations? 

(A) Chemical reactions involve the rearrangement of atoms about one another. 

(B) In an ordinary chemical reaction mass is not created nor destroyed. 

(C) Identical compounds are always composed of the same elements in the same 
proportion by mass. 

(D) Moles of chemical compounds are always conserved in balanced equations. 

(E) The number of moles of products formed is determined by the number of grams 
of the limiting reactant available. 



Data 

Tables 2-5 consist of data gathered for each of the four topics regarding the paired 
algorithmic and conceptual problems solved by subjects from each category of problem solvers. 
(In these tables "A" abbreviates algorithmic problem and "C" abbreviates conceptual problem, and 
"trans" is used to represent number of transitions between episodes of the problem-solving 
schema.) Table 2 presents the results for a typical algorithmic and conceptual problem for the 
topic of density solved by members of each subgroup available for study (i.e., expert, HA/HC, 
HA/LC, LA/LC), and also reports the means for all the novices as one group. (No data are 
reported for the members of the LA/HC group, because this research study produced no 
members classified as LA/HC.) Table 3 reports the same results for a typical algorithmic and 
conceptual stoichiometric problem; Tables 4 and 5 depict the results for the typical bonding and 
gas law problems, respectively. (Data from Table 3 are the focus of this paper, but Tables 2, 4, and 
5 are included for purpose of comparison.) The data exhibited in Tables 2-5 were collected from 
the incident identification graphs (see Figures 1-8) of the interviewed subjects* problem-solving 
schema and serve to illustrate composite examples of a typical subject found in each category. 
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Table 2 



Means of the Algorithmic and Conceptual Density Problems for each Category of Problem 
Solver 



Subgroup 


Atime 


Clime 


A trans 


C trans 


A rate 


Crate 


Expert 


2:28 


0:45 


3 


2 


1.3 


3.0 


HA/HC 


3:28 


3:06 


6 


7 


1.7 


1.6 


HA/LC 


3:47 


1:20 


7 


3 


1.8 


2.2 


LAAC 


351 


231 


8 


5 


2.1 


2.0 


Ai Novices 


3:46 


1:48 


7 


4 


1.9 


22 



Table 3 



Means of the Algorithmic and Conceptual Stoichiometric Problems for each Category of 
Problem Solver 



Subgroup 


Alime 


Clime 


A trans 


Ctrans 


A rate 


Crate 


Expert 


1:10 


1:35 


2 


2 


1.8 


1.6 


HA/HC 


2:15 


2:56 


5 


3 


2.2 


1.0 


HA/LC 


6:58 


4:09 


13 


8 


1.9 


1.9 


LAIC 


1027 


522 


22 


12 


2,1 


22 


AI Novices 


722 


420 


14 


8 


1.9 


1.8 



Table 4 



Means of the Algorithmic and Conceptual Bonding Problems for each Category of Problem 
Solver 



Subgroup 


Alime 


Ctime 


Atrans 


Ctrans 


A rate 


Crate 


Expert 


0:50 


0:15 


2 


1 


2.4 


4.5 


HA/HC 


1:35 


1:30 


2 


3 


1.3 


2.0 


HA/LC 


3:14 


1:04 


5 


3 


1.5 


2.8 


LAIC 


3:16 


1:48 


7 


5 


21 


2.8 


AI Novices 


3fl5 


1:18 


5 


4 


IB 


2.7 



Table 5 



Means of the Algorithmic and Conceptual Gas Law Problems for each Category of Problem 
Solver 



Suboroup 


Atime 


Ctime 


Atrans 


Ctrans 


A rate 


Crate 


Expert 


1:35 


0:40 


6 


1 


3.4 


1.5 


HA/HC 


3:26 


1:40 


9 


4 


2.6 


2.4 


HA/LC 


4:42 


2:44 


9 


4 


1.9 


1.5 


IMC 


601 


316 


14 


7 


23 


2.1 


AI Novices 


4S4 


2:46 


10 


5 


2.0 


1,8 
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Figure 1. Algorithmic episodic graph of a typical expert for stoichiometry. 
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Figure 2. Algorithmic episodic graph of a HA/HC novice on stoichiometry. 
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Figure 3. Algorithmic episodic graph of a typical HA/LC novice on stoichiometry. 
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Figure 4. Algorithmic episodic graph of a typical LA/LC novice on stoichiometry. 
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Conceptual episodic graph of a typical expert on stoichiometry. 
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Conceptual episodic graph of atypical HA/HC novice on stoichiometry. 
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Conceptual episodic graph of a typical HA/LC novice on stoichiometry. 
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Figure 8. Conceptual episodic graph of a typical LA/LC novice on stoichiometry. 



ERIC 



10 



10 

Figures 9 and 10 present data included in Tables 2-5, graphically. Figure 9 represents 
the total time needed to solve problems according to topic (i.e., density, stoichiometric, bonding, 
and gas laws) of the typical algorithmic problems for each group of problem solvers. Figure 1 0 
represents the total time needed by every group of problem solvers to solve each of the typical 
concepfual problems. In Figure 9 all times for the solving of an algorithmic-mode problem 
gradually increased as the problem-solving ability decreased, except in the cases of the time 
needed to solve an algc rfthmic-mode stoichiometric problem for the LC groups. In these cases 
there appears to be a much longer amount of time needed to solve an algorithmic-mode 
stoichiometric problem for the two LC groups, than for the other groups (see Figure 9). In Figure 
10 (except for the HA/HC conceptual density problem), there can be seen almost a linear 
relationship between problem-solving performance and the time required to solve a problem. 
(The anomaly seen in Figure 1 0 for the density problem of the typical HA/HC problem solver can 
be explained due to the fact that one of the two novices in that group experienced momentary 
confusion.) 

As the problem-solving performance decreased for the conceptual-mode problems on 
the topics of stoichiometry, bonding, and gas laws, the amount of time needed for subjects to 
solve a problem increased. Particularly note the curves on Figures 9 and 10 (reflective of the data 
presented in Table 3) for problem-solving time for each group regarding the topic of 
stoichiometry. As problem-solving ability decreased, there were dramatic increases in times 
required to solve both the algorithmic and conceptual stoichiometry problems. 
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Figure 9. Time required for a typical problem-solving schema for an algorithmic problem of 
each problem-solving classification. 
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Figure 10. Time required for a typical problem-solving schema for a conceptual problem of 
each problem-solving classification. 
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Figures 1 1 and 12 present graphs representing the data for the number of transitions for 
the typical algorithmic and conceptual problems (by topic), respectively. Figure 1 1 depicts the 
appropriate algorithmic data, and Figure 12 depicts data representative of a conceptual problem 
according to the problem topic. The same divergence of the curves on the graphs are seen as 
the problem-solving performance of the novices decline. Also as the performance level of the 
groups of novices decreased, the number of transitions required to solve the paired problems 
generally increased. The number of transitions required to solve an algorithmic and conceptual- 
mode problems appears somewhat clustered for all groups on all topics, except in the case of the 
stoichiometric problem. For the HA/LC and LA/LC groups of problem solvers, the number of 
transitions necessary for solving a conceptual-mode stoichiometric problem greatly exceeded that 
needed by members of the HA/HC and expert categories. However, the same general trend was 
established: the longer the amount of time required to solve an algorithmic or conceptual-mode 
problem (regardless of topic), the greater the number of transitions made by members of all 
experimental groups. 

Figures 13 and 14 graphically present the mean data for time and transitions for the 
novice groups only. Figure 13 depicts the average time required to solve each of the four pairs of 
algorithmic and conceptual problems, by topic of problem. In all cases the time required for a 
novice to solve an algorithmic-mode problem exceeded the time needed to solve the 
corresponding conceptual-mode problem. Figure 14 shows the average number of transitions 
made by the novices for algorithmic and conceptual-mode problems by topic. This graph is similar 
to the graph in Figure 13 in that in each case the number of transitions was always greater for the 
algorithmic problem than for the conceptual problem. 
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Ffcare t r. Number of transitions for a typical algorithmic problem of each classification of 
problem solver by type of problem. 
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F/gure 1 2. Number of transitions for a typical conceptual problem of each classification of 
problem solver by type of problem. 
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Figure 14. Number of transitions required to solve each type of paired problems. 
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Interpretations of the graphed data from Tables 2-5 deems necessary a more in depth 
analysis of the problem-solving ability of all subjects regarding procedures used to solve 
stoichiometric problems. Since both the algorithmic and conceptual-mode stoichiometric 
problems presented all groups increased difficulty, transcripts from the think-aloud interviews of 
each individual were reviewed. Correct solutions to the algorithmic-mode problem were obtained 
by nearly all interviewed subjects (i.e., all experts and both HA/HC novices solved the algorithmic- 
mode problem correctly, along with all but three of the remaining 17 LC novices). In light of the 
fact that only three interviewed subjects failed to obtain the correct solution to the algorithmic- 
mode problem, the remaining part of the investigation focuses on the conceptual understanding 
exhibited by individuals on the topic of stoichiometry. 

Of the 22 interviewed subjects, 16 failed to concluded the correct solution to the 
conceptual-mode problem. Of the 16 subjects who obtained an incorrect solution to the 
conceptual-mode problem, one was an expert, ten fell into the HA/LC category, and all five LA/LC 
novices arrived at an incorrect solution to the given problem. The frequency of the incorrect 
choices is reported in Table 6. 

Table 6 

Frequency of incorrect responses to the conceptual-mode stoichiometric problem by problem- 
solving category 



group 


n 


A 


B 


C 


D* 


E 


Expert 


2 










1 


HA/HC 


2 












HA/LC 


13 


1 




5 




4 


LA/LC 


5 




2 


2 




1 



*correct solution 
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Responses from the think-aloud interviews given by subjects in the high conceptual 
categories (expert and HA/HC groups) who correctly solved the conceptual-mode stoichiometric 
problem, rarely stated anything after reading the problem outside of simply giving the answer to 
the problem as choice W D M . When asked to justify why they had selected choice M D M , the answers 
given by all three subjects were similar. They basically responded they knew that moles were not 
conserved in balanced chemical equations and therefore this must be the con ect choice (or that 
choice "D M is the given statement that was not associated with ail balanced equations). (See 
example problem on stoichiometry.) For the other 16 subjects who incorrectly solved the 
problem, all attempted to obtain the correct solution by a process of attempting to eliminate 
(sometimes haphazardly) four of the five possible choices. Figures 3, 4, 7, and 8 clearly depict 
that these subjects' ability to arrive at a solution to the problem in a logical, sequential manner is 
discouraging. An increased amount of time and numerous transitions between episodes are 
needed for these students to arrive at any solution. Both choices M C M and M E M contributed as 
competing distracters for the correct solution to the problem (see Table 6). Answer "C" is a 
restatement of the Law of Definite Proportions and choice "E M defines a limiting reactant. 

When average reading times for LC novices are compared to the typical subjects in the 
HC groups, there is almost a two-fold increase. Where the typical expert and HA/HC novice spend 
only minimal time defining issues within the problem, the subjects from the LC novice groups 
exhibit seemingly erratic logic between the episodes of define and set up (see Figures 5-8). The 
average number of transitions required while the conceptual-mode problem was solved for the 
expert group is 2, the HA/HC group is 3, the HA/LC group is 8, and the LA/LC is 12 (see Table 3). 
Likewise, dramatic increases in the number of transitions between episodes (i.e., expert * 2, 
HA/HC = 5, HA/LC = 13, and LA/LC =* 22) was seen in the incident identification graphs for the 
algorithmic-mode problem (see Table 3 and Figures 1-4), but these problems were eventually 
solved correctly by most individuals interviewed. 
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Results and Conclusions 
These results led to the acceptance of research hypotheses, Hi and H2, of the research 
question. There were similarities and differences in the ways different groups of problem solvers 
solved both algorithmic and conceptual-mode stoichiometric problems typical of the ones found 
in introductory chemistry courses. Not only did LA/LC problem solvers take more time to solve 
both algorithmic and conceptual-mode problems, than did members of the other categories, but 
also they required more transitions between the episodes of the problem solving schema than did 
members of the other categories of problem solvers. Other results indicated that stoichiometric 
problems (both modes) were most difficult for the novices in the LC groups. Not only was more 
time required, but also more transitions were needed over those required to solve paired 
problems on the other three topics. An implication of this observation is that there must have 
been some confusion among the members of this category of novices as they solved the 
algorithmic and conceptual-mode stoichiometric problems. Gas law problems (whether algorithmic 
or conceptual) ap ^red to be the second most difficult problem-solving topic for the LC novices. 
This conclusion was based on the increased times required and the greater number of transitions 
needed to solve problems on this topic over that needed to solve problems on topics of density 
and bonding. 

As observed from the graph of Figure 9, there is a large divergence in the times needed 
to solve an algorithmic-mode stoichiometric problem by members of the different problem-solving 
categories. This same separation is evident in the corresponding conceptual-mode graph (see 
Figure 10). This observation emphasizes that students with low-algorithmic and low-conceptual 
performance find problems on stoichiometry to be much more difficult than do members of the 
other groups. This result is revealed again by the graph for the number of transitions needed to 
solve algorithmic and conceptual-mode stoichiometric problems (see Figures 1 1 and 1 2). 
However, regardless of the problem topic, all groups tended to solve the conceptual-mode 
problem of the pair more quickly and with fewer transitions (see Figures 13 and 14) than the 
corresponding algorithmic-mode problem. This reaffirms that conceptual understanding is more 
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dependent upon a known definition or relationship which can be more quickly accessed by the 
solver. 

Regardless of problem topic, the time required for a novice to solve an algorithmic-mode 
problem exceeded the time needed to solve the corresponding conceptual-mode problem. As 
subjects' problem-solving ability decreased, the times and number of transitions needed for each 
group of problem-solvers to complete the problems gradually increased on topics of density, 
bonding, and gas laws regardless of whether the problem was of a computational or conceptual 
nature, but the divergence of the curves of the stoichiometric problems exhibited a much steeper 
slope. This finding is consistent for both the algorithmic and conceptual-mode problems. These 
results also reaffirm that conceptual understanding is more dependent upon the subjects' ability 
to access prior knowledge regarding chemical interactions and relationships than is their ability to 
compute answers to algorithmic-mode problems. In light of the fact that over 85% of the sample 
tested succeeded in answering the computational problem correctly, we can assume that lack of 
mathematical ability may only play a small role in overall student success in introductory chemistry. 
The major portion of the novices' difficulty in solving stoichiometric problems may lie in their lack of 
conceptual understanding. Consequently, more time and effort should be exerted by instructors 
to assure that introductory-level students have sufficient practice to succeed at solving these 
more difficult conceptual problems. 

Implications 

Another aspect of the selected conceptual problem which must be addressed is the 
inclusion in this problem on stoichiometry the many facts and numeral values which were not 
necessary in order to arrive at a satisfactory solution (i.e., all the introductory information in the first 
two statements). In the real world most problems present themselves with many variables which 
need to be identified as pertinent, or not, to obtaining a solution. Some variables may prove to be 
important and some may not, so problem solvers must learn to differentiate important facts that are 
needed from those which simply contribute to the general data base. 
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There are several techniques that can be used to aid our students In solving conceptual- 
mode problems relevant to the study of introductory chemistry: (1) present lecture information to 
students in a format representative of the two distinct problem-solving modes; (2) design 
examinations to reflect emphasis not only on computations, but also on basic chemical concepts; 
(3) include in examinations/problem sets "extra" information that allows the student to explore 
more than the traditional "plug n' chug" mentality; (4) train students in the use of the language of 
chemistry by encouraging in-class assignments that invite a collaborative effort between the 
instructor and the students and among the students themselves; and (5) find ways to introduce 
the more conceptual aspects of chemistry to younger/less experienced students. 
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